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ABSTRACT 

The mean-square error of reduced-order linear 
state estimators for continuous-time linear systems 
is investigated. Lower and upper bounds on the 
minimal mean-square error are presented. The 
bounds are readily computable at each time-point 
and at steady state from the solutions to the 
Ricatti and the Lyapunov equations. The usefulness 
of the error bounds for the analysis and design of 
reduced-order estimators is illustrated by a prac- 
tical numerical example. 


INTRODUCTION 

The need for order reduction in linear state 
estimation stems from the difficulties associated 
with the numerical implementation of a full-order 
optimal estimator for a high-order system. Several 
suboptimal reduced-order estimator design tech- 
niques have been suggested in the literature (e.g., 
[1]). An optimal mean-square solution for time- 
invariant linear systems at steady state was sug- 
gested by Bernstein and Hyland [2). Time-varying 
systems were considered by Sims and Asher [3]» who 
treated the case in which only some of the state 
variables are of interest. For the more general 
time-varying case in which reduced-order estimation 
of the full state vector is desired, there appears 
to be no tractable optimal solution. 

In this paper we first derive a lower bound on 
the mean-square error of any state estimator having 
a specified reduced order, for possibly time- 
varying system. The lower bound is computable for 
any time-point and, in the case of a time- invar iant 
system, for steady state, using standard algo- 
rithms. Its values for all reduced orders are 
obtained simultaneously and can be readily used to 
lower-bound the order, which is necessary to 
achieve a specified performance level. The mean- 
square error for a given reduced-order estimator 
can be easily calculated. It can be used to eval- 


uate the estimator quality relative to that of 
alternative designs, to the lower-bound and to the 
optimal, full-order estimator. It constitutes an 
upper bound on the minimal mean-square error, which 
can be used to eliminate nonoptimal solutions 
(e.g., local extrema) in an optimization process. 


LOWER BOUND ON MEAN-SQUARE ERROR OF A 
REDUCED-ORDER ESTIMATOR 

Consider the system 


i(t) z A(t)x(t) 

♦ B(t)w(t) 

Oa) 

y(t) z C(t)x(t) 

+ v(t) 

( 1b) 


where x(t) e R n , y(t) e R m , x(0) is Gaussian 
with E{ x(0) ) r 0, E{x(0)x(0) T } = P(0), and w(t) 
and v(t) are mutually uncorrelated Gaussian white- 
noise processes, uncorrelated with x(0), with 
E{w( t ) ] z E(v( t) } z 0, E(w( t )w( t) T } = Q(t) , 
E(v(t)v(t) T } z R(t) , and E(w(t)v(t) T } z S(t). An 
estimator of reduced order r < n for x(t) has 
the form 

q(t) = F(t)q(t) ♦ G(t)y(t), q(0) given (2) 

where q(t) e R r , so that an estimate of x(t) can 
be obtained by some linear transformation 

S(t) = H( t)q( t) , £(t) e R n (3) 

It follows that x(t) has, at most, rank r, that 
is, its covariance matrix has a rank smaller than 
or equal to r. 

Let us denote by Y ( t ) = { y ( s ) , s < t) the 
o-algebra of the past observations and by x(t) the 
conditional expectation of x(t), given Y(t). 
Employing the orthogonality principle, the mean- 
square error in the estimate x(t) can be written 
as 
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which implies 


E{(x(t) - x(t)) T (x(t) - x(t))> 

= E ( [ { x( t) - x(t)) + (x(t) - x(t))] T 
•[(x(t) - x(t)) + (x(t) - x(t))]) 

= E{( x( t) - x(t)) T (x(t) - x(t))} 

+ E((x(t) - x(t)) T (x(t) - x(t))} (H) 

Let us denote by .W(t) the matrix whose columns are 
.the eigenvectors of E{x(t)x(t) ) arranged in 
accordance with the corresponding eigenvalues in 
descending order of magnitude. Let us separate 
W(t) as W(t) = [U( t ) V( t ) ] where U(t) has r 
columns. Then 

T- P (t)P l 

W(t)x(t) : J a.s. 

where x(t) is a vector of dimension r, yielding 
P (t>r l - r 

x(t) = W( t) I I = U(t)x(t) r a.s. (5) 

Since W(t) is unitary, we have 

W(t)W(t) T = U(t)U(t) T + V(t)V(t) T = I 

Hence, 

E{(x(t) - x( t) ) T (x( t) - x(t))) 

= E{x( t) - U(t)x(t) r ) T (x(t) - U(t)x(t) r )> 

= {(x(t) - U( t )x( t ) r ) T W( t ) 

• W(t) T (x(t) - U(t)x( t) r )} 

yielding 

E<(x(t) - x(t) ) T (x(t) - x( t) ) } 

= E{ ( x( t ) - U(t)S(t) r ) T U(t) 

• U(t) T (x(t) - U(t)x(t) r )} 

+ E((x(t) - U(t)x(t) r )) T V(t) 

• V(t) T (x(t) - U(t)x(t) r )> (6) 


E{( x( t ) - x(t)) T (x(t) - x( t ) ) > 

> E{(x( t) - U(t)x(t) r ) T V( t) 

• V(t) T (x(t) - U(t)x(t) r )} 

= E{x(t) T V(t)V(t) T x(t)l (7) 

Noting that 

V(t)V(t) T = I - U(t)U(t) T 

= (I - U(t)U(t) T )(I - U(t)U(t) T ) 

we see that 
E{x( t) T V( t)V( t) T x( t) } 

= E{x(t) T ( I - U(t)U(t) T )(I - U(t)U(t) T x(t)} 

= tr {( I - U(t)U(t) T )n(t)(I - U(t)U(t) T )} 

= tr{(n(t) 1/2 - U(t)U(t) T n(t) 1/2 ) 

• ( n( t) 1/2 - U(t)U(t) T n(t) 1/2 )> ( 8) 

where n(t) = E{x(t)x(t) T } . 

Let Xj(t) t i = 1,...,n denote the eigen- 
values of n(t) arranged in descending order of 

T 1 /? 

magnitude. Since clearly U(t)ll(t) n ( t ) has, 
at most, rank r, it follows from a well-known 
result (e.g., [4], p. 63) that 

n 

E{x(t) T V(t)V(t) T x(t)} > £ x i (t) 

i=r+1 

with equality if and only if 
T(t) T U(t)U(t) T n(t) 1 / 2 T(t) 

r diag(X 1 (t) 1/2 X r (t) 1/2 ,0,...,0) 

It follows that e (t) is bounded below by 

n 

e*(t) = tr (P( t) } ♦ Y x i (t) (9) 

i=r+1 

2 

Since e Q (t) is a lower bound on the mean-square 
error of any estimate of x(t) having rank r, it 
is a lower bound on the mean-square error of any 
reduced-order state estimator of order r for the 
system ( 1 ) . 

We note that P(t) satisfies the Riccati 
equation 



P(t) = A(t)P(t) + P( t ) A( t ) T + B( t )Q( t )B( t ) T with 


- [P(t)C(t) T ♦ B(t)S(t)]R(t)' 1 
• [P(t)C(t) T * B(t)S(t)] T (10 ) 


and that n(t) satisfies the Lyapunov equation 
ii(t) s A(t)n(t) ♦ n(t)A(t) T 

+ P(t)C(t) T R(t)' 1 C(t)P(t) (11 ) 


with n(0) = P(0) given. Methods for solving these 
equations can be found in the literature (e.g., 
[51). The lower bound can then be calculated as a 
function of time by solving (10) and (11). In the 
time-invariant case, the steady-state value of the 
lower bound can be obtained by substituting in (9) 
the steady-state solutions of (10) and (11); that 
is, the solutions of the algebraic Riccati and 
Lyapunov equations, respectively. 


Once the matrix n(t) is diagonalized, the 
lower bound is obtained simultaneously for all 
reduced orders by simply including the correspond- 
ing eigenvalues in (9). It can be seen that the 
lower bound is a monotone decreasing function of 
the order, which attains its minimal value, 
tr{P(t)J, for r = n. The error can be used then 
to determine a lower bound on the order, needed for 
maintaining the estimation error below a certain 
level. Specifically, if the maximal acceptable 
error is a(t), then a lower bound on the order is 


r . 
min 


min 


,r: ^ X.(t) < o(t) - tr[P( t) J 


* 


i=r+1 


J, 


( 12 ) 


This is well illustrated by the closing example. 


UPPER BOUND ON MEAN-SQUARE ERROR OF A 
REDUCED-ORDER ESTIMATOR 

Suppose that a reduced-order estimator of 
order r < n is given by (2). Then the covariance 
of the estimation error, given by 

n(t) : E{[x(t) - H(t)q(t) ][x(t) - H( t)q(t) ) T > 

(13) 



The latter satisfies the equation 

x(t) = F(t)x(t) ♦ w(t) ( 16) 

where 


A(t) 0 

,G(t)C(t) F(t)_ 


(17) 


and w(t) is a zero mean white-noise process with 


cov(w( t) ] 


Q(t) 0 

. 0 G(t)R(t)G(t) T . 


( 18 ) 


It follows that r(t) satisfies the Lyapunov 
equation 

r(t) = F(t)r(t) + r(t)F(t) T + cov(w(t)| (19) 

The mean-square error of the reduced-order esti- 
mator is 


e 2 (t) = tr (Q( t) } (20) 

p 

It should be noted that e (t) at any time t is 
computable by standard algorithms for solving 
(19). When the system and the estimator are time- 
invariant, e 2 ( t ) attains a limit value e 2 if 
both A and F have their eigenvalues in the left 
half-plane. 

Minimizing the mean-squared estimation error 
with respect to the reduced-order estimator param- 
eters is a difficult optimization problem. In 
order to minimize the error at time t, the optimal 
matrix functions {F(t),G(t),H(t),0 < x < t} must be 
found [31. For a different time-point, the entire 
matrix functions must be found anew. Under the 
assumptions of time-invariance and stability, the 
problem may be formulated as one of minimizing the 
steady-state value of the mean-square error e 2 . 
Even in this form the optimization problem is quite 
difficult, and its solution requires the simulta- 
neous solution of a set of matrix equations [2). 


can be calculated from 


where 


n(t) = [I - H(t)]r(t) 


1 1 

L-H(t)J 


(14) 


r(t) z EUUJxU) 1 } 


(15) 


Clearly, for any specified estimator of the 
form (2), the mean-square error (20) constitutes an 
upper bound on the minimal attainable error. It is 
a useful measure of the quality of the given esti- 
mator both in absolute terms and in comparison with 
alternative reduced-order estimators and with the 
optimal, full-order one. In the previous section 
we derived a global lower bound on the mean-square 
error. A comparison of the specific upper bound, 


corresponding to a given estimator, to the global 
lower bound would provide information not only on 
the domain of the minimal possible error, but also 
on the quality of the given estimator as a simple, 
easy-to-obtain alternative to the optimal reduced- 
order estimator, which is difficult, if not impos- 
sible, to derive. 

A simple reduced-order estimator for the sys- 
tem (1) is now derived. We first note that the 
minimum variance estimate x(t) of x(t) satisfies 
the equation 

x(t) = [ A( t) - K( t)C( t) ]x( t ) ♦ K(t)y(t) 


The resulting estimator of order r produces, by 
means of (2*0, optimal-state extimates, without 
degrading the performance of the full-order Kalman 
filter. 

Next, suppose that there are more than r 
nonzero eigenvalues of n(t). Still, it is desired 
to reduce the estimator order to r. Eliminating 
the n - r components of z(t) with smallest 
variances, which are then approximated by zero, the 
resulting reduced-order estimator is (25), which we 
now write as 

q(t) = F(t)q(t) ♦ G(t)y(t) (26) 


where K(t) is the Kalman gain, given by 

K(t) = [P(t)C(t) T ♦ B( t)S(t) ]R(t) _1 
It follows that the vector 

z(t) s T(t) T x(t) 


with 

F(t) = [I p 0]T( t ) T [ A( t ) + T( t)f ( t) T 
- K(t)C(t)]T(t) 



satisfies the equation 


and 


z(t) = T( t) T [ A( t) ♦ T(t)T(t) r - K(t)C(t)]T(t)z(t) 


+ T(t) l K(t)y(t) 


( 21 ) 


Suppose that the matrix n(t) has r non-zero 
eigenvalues. Then 


A(t) = E{z( t )z( t) 1 } = 


A t (t) 


L o 


x (t) 
r 0 


( 22 ) 


This means that the values of z r+ ? ( t) , . . . ,z n ( t) 
are zero, with probability 1. These elements can 
then be eliminated from the vector z(t). Defining 


q(t) = [I p 0 ]z( t ) = (I p 0 |T( t ) 1 x( t ) (23) 

where I p is the identify matrix of dimension r, 
we have 


x ( t ) = T(t) 


Cl 


q(t) 


(24) 


It follows that q(t) satisfies the equation 
q(t) = [I p 0 ]T( t) T [ A( t ) + T( t)T ( t) T 


- K(t)C(t)]T(t) 


.0 . 


q(t) 


♦ [I p 0)T( t ) *K( t )y ( t ) 


(25) 


G(t) = [I p 0 )T( t ) T K( t ) 

It can be seen that the difference between the 
reduced-order estimator (26) and the optimal lies 
in the elimination of the term 

[o I n _ r ]T( t) T [A(t) + T(t)i(t) T 

0 

-K(t)C(t)T(t) 

I 

- n ~ r „ 


from the dynamics matrix. In the time- invariant 
case the steady-state value of the bound can be 
obtained from the corresponding steady-state equa- 
tions. In this case, the matrix A - KC has all 
its eigenvalues in the left half-plane and, conse- 
quently, so does the matrix T T (A - KC) T T. This 
does not guarantee, however, that F has all its 
eigenvalues in the left half-plane, which can nev- 
ertheless be checked numerically for the given 
system. In the time-varying case the calculation 
of the upper bound requires integrating (10), (11), 
and (19), and calculating T(t) at each time- 
point. The error owing to the above eliminated 
term may diverge, and the upper bound may not be 
sufficiently tight to be useful. This, however, 
can be checked numerically by comparison with the 
lower bound. 


EXAMPLE 

The lower and the upper bounds for a seventh- 
order linear model for a power system, given by 



Mahalanbis and Ray [6], are displayed in Fig. 1. 

It can be seen that for orders greater than 2 the 
upper and the lower bounds have nearly the same 
values. This means that for orders greater than 2, 
the simple reduced-order estimator can be used 
instead of the optimal reduced-order one, which is 
difficult to derive. The fact that for orders 
greater than 3, both bounds are close to the mean- 
square error of the full-order Kalman filter 
implies that a simple reduced-order estimator of 
order 4 can be used instead of the Kalman filter, 
with an accuracy loss of less than 3 % of the opti- 
mal error. 


CONCLUSION 

The design of a reduced-order state estimator 
which gives a minimal mean-square error is a diffi- 
cult, if not practically impossible, task. Bounds 
on the mean-square error are useful for analysis 
and design purposes. In this paper, we derived a 
lower bound on the mean-square error of any esti- 
mator having a specified reduced order for a given 
system. The bound is readily computable for any 
reduced order at any time-point and, for time- 
invariant systems, at steady state. It is useful 
for evaluating, by comparison, any given design and 
for lower-bounding the order needed for a specified 
accuracy. The latter application offers a consid- 
erable reduction in the computation load of any 
reduced-order estimator design scheme. A specific 
reduced-order estimator was proposed, from which an 
upper bound on the minimal mean-square error was 
calculated. The usefulness of the bounds in 



Ik 


Fig. 1 Lower and upper bounds on min 
example. 


reduced-order estimator analysis and design was 
illustrated by a practical example. 
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